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summary wil, 


Across the United States, and in the states served by the Regional Educational Laboratory 
West (REL West) in particular, there is widespread concern about how to successfully 
educate the growing number of English language learner (ELL) students, especially those 
identified as long-term ELL students and those identified as reclassified fluent English profi- 
cient (RFEP) students who struggle to score at passing levels on state English language arts 
(ELA)/reading content tests (Horwitz et al., 2009; Olsen, 2010; Quality Counts, 2009). 


This study, which focuses on ELL students in Utah, is one of a series of three companion 
studies that seek to help Arizona, Nevada, and Utah identify the characteristics of long-term 
ELL students and Struggling RFEP students. For this study, we defined long-term English 
language learner students as students who, during the six school years of the study, never 
scored at or above the levels required on Utah’s English language proficiency (ELP) test 

to be reclassified as fluent English proficient. We compared these long-term ELL students 
to their ELL peers who did score at or above the levels required on Utah’s ELP test to be 
reclassified as fluent English proficient, whom we refer to as reclassified fluent English profi- 
cient (RFEP) students. Struggling RFEP students were defined as ELL students who met the 
state’s ELP requirements for RFEP but did not pass the state ELA or reading content test by 
the end of year 6 of the study. We compared these Struggling RFEP students to their RFEP 
peers who did pass the state ELA or reading content test, whom we refer to as Transitioned 
RFEP students. 


The study examined student data from 2006/07-2011/12 to address three research questions: 


» What proportion of ELL students fit the study’s definition of long-term ELL 
students by the end of the six years? What proportion of ELL students fit the study’s 
definition of Struggling RFEP students by the end of the six years? 


» What are the characteristics of long-term ELL students? How are these characteris- 
tics different from those of RFEP students? 


» What are the characteristics of Struggling RFEP students? How are these character- 
istics different from those of the RFEP students who passed their ELA content test at 
least once by the end of the six years? 


The study followed three cohorts of ELL students in Utah: a grade K cohort, who started 
kindergarten in 2006/07; a grade 3 cohort, who started grade 3 in 2006/07; and a grade 6 
cohort, who started grade 6 in 2006/07. By examining these three cohorts in Utah over six 
years (2006/07-2011/12), this study found the following: 
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Subgroup proportions 


» At least two-thirds of the ELL students scored at or above Utah’s required ELP level 
to meet RFEP criteria. 


» Among RFEP students, across all three grade-level cohorts, at least 90 percent 
passed their ELA content test at least once (thereby becoming Transitioned RFEP 
students). 


» Among long-term ELL students, at least 30 percent from each of the three grade- 
level cohorts passed the ELA content test at least once. 


Subgroup characteristics 


» Long-term ELL students, when compared to their more successful RFEP peers, had 
higher percentages of eligibility for free or reduced-price lunch (FRL), eligibility 
for individualized education program (IEP) services, and male students. 


» In general, Struggling RFEP students, when compared to their more successful 
Transitioned RFEP peers, had higher percentages of eligibility for FRL, eligibility 
for IEP services, and male students. 


» Compared to their more successful RFEP and Transitioned RFEP peers, long-term 
ELL students and Struggling RFEP students also had higher percentages of students 
with lower ELP levels during the first study year. 


This report’s findings on Utah ELL students suggest several additional questions worth 
further investigation. For example, why did this study see patterns that differ from other 
research regarding the achievement of reclassification as fluent English proficient by grade 
levels? Findings from the research literature generally show that it is harder for older 
students to progress to English proficiency and to pass content tests than for younger 
students to do so. Yet, in Utah, the percentages of students across grade-level cohorts who 
scored at the level required for RFEP and of the RFEP students who passed the ELA content 
test at least once (Transitioned RFEP) were close to constant. Another question is what can 
be done, or what has been found effective, in assisting the lower-achieving ELL student 
subgroups to close the achievement gaps? Further exploration of these questions would help 
identify which groups of ELL students take longer than expected to reach English language 
proficiency and how interventions could be targeted to assist them. 
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Why this study? BL li 


Across the United States, and in the states served by the Regional Educational Laboratory 
West (REL West) in particular, there is widespread concern about how to successfully 
educate the growing number of English language learner (ELL) students, especially those 
identified as long-term ELL students and those identified as reclassified fluent English 
proficient (RFEP) students who struggle to score at passing levels on state English language 
arts (ELA)/reading content tests (Horwitz et al., 2009; Olsen, 2010; Quality Counts, 2009). 
The members of REL West’s English Learner Alliance—the state departments of education 
from Arizona, Nevada, and Utah—requested a study of the characteristics of long-term ELL 
students and Struggling RFEP students in each alliance state (resulting in a series of three 
companion state reports) in order to better understand how these two groups of low-per- 
forming ELL students may differ from the ELL student population as a whole in each state. 
In this study, Struggling RFEP students are defined as ELL students who met the state’s 
English language proficiency (ELP) requirements for RFEP, but who did not pass the state 
ELA or reading content test during the course of the study. 


Currently, the English Learner Alliance states of Arizona, Nevada, and Utah have not 
analyzed the characteristics of their long-term ELL students and Struggling RFEP students, 
but they are eager to do so. Specific requests have come from staff in each of the three state 
departments of education, who, individually and as a group, developed this research agenda 
with REL West. 


The English Learner Alliance members intend to use the study’s findings to inform early 
interventions targeted at ELL students whose characteristics fit patterns of ELL students 
who are more likely to become long-term ELL students or to remain Struggling RFEP 
students. Further, they would like to develop a shared understanding of their long-term ELL 
students and Struggling RFEP students, in order to promote cross-state collaboration in 
developing effective interventions and programs. The English Learner Alliance would also 
like to use this project as an initial model for similar follow-up projects, including longitu- 
dinal analyses of ELL student progress in ELP tests and academic subject-matter tests. 


While there is widespread concern about how to successfully educate the growing number 
of ELL students, especially those identified as long-term ELL students and/or Struggling 
RFEP students, there do not appear to be any studies of the characteristics of these two 
groups of students that might enable states to better target existing programs and interven- 
tions (see, e.g., Burr, Haas, & Geary, 2013) by basing them on the particular needs of those 
two groups. This Utah study, along with the two companion state reports from Arizona and 
Nevada, will provide this needed information, based on data from all three states. 
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Box 1. Defining categories of English language learner students for 
this study 


Long-term English language learner students 


Students who never scored at or above the required levels on their state English language 
proficiency (ELP) test to be reclassified as fluent English proficient during the six years 
of the study. 


Reclassified fluent English proficient (RFEP) students 


Students who scored at or above the required levels on their state ELP test to be reclassi- 
fed as fluent English proficient during the six years of the study. 


Struggling RFEP students 


RFEP students who met the ELP classification requirements as fluent English proficient, 
but did not pass their state’s English language arts (ELA) or reading content test during 
the six years of the study. 


Transitioned RFEP students 
RFEP students who passed their state’s ELA or reading content test at least once during 
the six years of the study. 


ELL achievement and student characteristics 


ELL students, as a group, tend to lag behind native English speakers in their rates of academic 
achievement (Kindler, 2002; Massachusetts Department of Elementary and Secondary 
Education, 2012; Olsen, 2010; Ruiz-de-Velasco & Fix, 2000; Short & Fitzsimmons, 2007). 
This is due, in large part, to the need of ELL students to simultaneously learn English and 
content knowledge (Genesee, Lindholm-Leary, Saunders, & Christian, 2005). However, both 
ELL students and former ELL students (i.e., RFEP students) are diverse groups with different 
strengths and needs, depending on a number of characteristics (Kindler, 2002). 


Most states and districts collect data on several characteristics that appear to be related to 
academic achievement for ELL students specifically and students generally. These character- 
istics include 


» poverty status (Goldenberg, 2008; Mulligan, Halle, & Kinukawa, 2012; Rathbun & 
West, 2004; Roberts & Bryant, 2011); 


» disability status (Liasidou, 2013; McCardle, McCarthy-Mele, Cutting, Leos, & 
D’Emilio, 2005; Nguyen, 2012); 


» gender (Perie, Moran, & Lutkus, 2005); 


» initial ELP when ELL students first enroll in school (Cook, Linquanti, Chinen, & 
Jung, 2012; Collier, 1989, 1992; Halle, Hair, Wandner, McNamara, & Chien, 2012); 
and 


» grade level (Genesee et al., 2005). 
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At present, the research literature describes the impact of each of these student characteris- 
tics in the following ways. 


Poverty status 


Research on ELL students generally shows that their socioeconomic status (SES) has an 
impact on academic achievement. For instance, ELL students from homes with lower SES 
generally score lower on academic content tests and are less likely to score proficient on ELP 
tests than their higher-SES peers. Two analyses of the Early Childhood Longitudinal Survey 
from the kindergarten class of 1998/99 support these conclusions about the impact of SES on 
ELL students’ academic achievement. A greater percentage of kindergarten students whose 
primary language was not English than of students who entered kindergarten as English 
proficient came from families whose incomes were below the federal poverty threshold; in 
addition, ELL students who came from families with incomes below the federal poverty 
threshold scored lower on reading, mathematics, and science tests than ELL students whose 
families had incomes above the federal poverty threshold (Mulligan et al., 2012). A similar 
conclusion was reached by Roberts and Bryant (2011), who found that SES was more salient 
than primary language in explaining the mathematics achievement of ELL students; they 
found that ELL students with higher SES scored higher on mathematics assessments than 
ELL students with lower SES. 


ELL students with disabilities 


There do not appear to be any studies that address the impact of specific disabilities on an 
ELL student’s academic progress in the United States. However, one meta-analysis review 
describes the likely impact in Canada (Lipka, Siegel, & Vukovic, 2005). The authors of 
this meta-analysis reviewed published studies of ELL students in Canada, with the goal of 
understanding the reading development of ELL students and the characteristics of reading 
disabilities in this population. Phonological processing, syntactic awareness, and working 
memory of ELL students with and without reading disabilities were compared to those of 
native English-speaking students with and without reading disabilities. For each of the 
three ELL native language-specific (Portuguese, Italian, and Arabic) studies, students (ages 
9-14) with and without reading disabilities were compared within their native-language 
groups and to their native English-speaking peers (AbuRabia & Siegel, 2002; Da Fontoura 
& Siegel, 1995; D’Angiulli, Siegel, & Serra, 2001). The findings were consistent across the 
three studies: within each native-language group, ELL students with reading disabilities had 
much lower scores than ELL students without reading disabilities, while ELL students with 
reading disabilities had similar levels of performance to native English-speaking students 
with reading disabilities. 


Gender 


There do not appear to be any studies that examine the impact of gender on an ELL 
student’s academic progress. However, there are studies that describe differences in academic 
achievement by gender among the K-12 general population. Over the past decade, results 
from the National Assessment of Educational Progress (NAEP) have shown small but 
persistent mathematics gender disparities favoring males at grades 4, 8, and 12, with gaps of 
roughly 0.1 standard deviations, or the equivalent of a few months of schooling (McGraw, 
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Lubienski, & Strutchens, 2006; Perie, Moran, & Lutkus, 2005); in contrast, reading achieve- 
ment data from the 2005 and 2007 NAEPs reveal that females outscored males by less than 
0.2 standard deviations at grade 4 but more than 0.3 standard deviations at grades 8 and 12 
(Perie et al., 2005). A more recent study (Robinson & Theule, 2011) found generally similar 
patterns. Using K-8 national longitudinal data, the authors investigated males’ and females’ 
achievement in mathematics and reading, including when gender gaps first appeared, 
whether the appearance of gaps depended on the metric used, and where the achievement 
distribution gaps were most prevalent. The authors found no mathematics gender gap in 
kindergarten, except at the top of the distribution; however, females throughout the distri- 
bution lost ground in elementary school and regained some in middle school. In reading, 
gaps favoring females generally narrowed as the grade level increased, but widened among 
low-achieving students. 


Initial English language proficiency and grade level 


Research generally shows that ELL students tend to make greater year-to-year ELP and 
academic content improvement in the lower grades than they do in the higher grades, when 
both age groups start at the same ELP (Cook, Wilmes, Boals, & Santos, 2008; Grissom, 
2004; Kieffer, 2008, 2010, 2011; Salazar, 2007). Thus, ELL students are more likely to take 
longer to progress from the Intermediate ELP level to the fully Proficient ELP level in 
higher grades, such as high school, than ELL students starting at the same ELP level in the 
elementary school grades. In addition, this difference in lower and higher grade levels’ rate 
of progress in ELP tends to become greater as the level of ELP increases (Cook et al., 2008; 
Garcia, 2003; Halle et al., 2012). In other words, a student at a low grade level and a low 
ELP level (such as a kindergarten student at the Pre-emergent ELP level) would typically 
have a much higher rate of progress in achieving RFEP (that is, scoring Proficient on the 
ELP test) than a student at a high grade level and high ELP level (such as a grade 11 student 
at the Intermediate ELP level). 


For progress on subject-matter content tests, there do not appear to be any research stud- 

ies that specifically describe the characteristics of Struggling RFEP students. There are, 
however, related research studies that provide partial insights into the characteristics asso- 
ciated with the achievement of ELL students on academic content tests. For example, the 
length of time for initially designated ELL students to reach the 50th percentile on academic 
achievement tests ranged from four to ten years and depended on several factors: the level of 
the student’s schooling in his or her first language; the student’s initial level of English profi- 
ciency; the type of language program (for example, two-way bilingual, sheltered English 
immersion); and the subject being tested (for example, mathematics knowledge appeared to 
transfer into the English language assessments more readily than reading and writing skills) 
(Collier, 1989, 1992). 


Characteristics examined in this study 


This study describes several characteristics of Struggling RFEP students—initial level of 
ELP, as well as gender, eligibility for FRL, eligibility for IEP services, and grade level—as 
they relate to achievement on Utah’s ELA content test. State-level data were not available in 
Utah (or in Nevada or Arizona) on ELL students’ proficiency in their first language, or on 
the types of language programs that ELL students are participating in. 
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This study addresses the gaps in the research literature by providing descriptions of the 
initial characteristics! of Utah ELL students who became either long-term ELL students or 
Struggling RFEP students and of any differences in the initial characteristics between these 
groups and their more successful counterparts, RFEP students as a whole, and Transitioned 
RFEP students. 
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What the study examined aE 


This study on ELL students in Utah—like its two companion reports on ELL students in 
Nevada? and Arizona—is a descriptive analysis of the progress on state assessments and 
characteristics of four categories of ELL students: long-term ELL students, RFEP students, 
Struggling RFEP students, and Transitioned RFEP students. Utah’s state testing process is 
described in box 2. 


In this study, long-term ELL students were defined as ELL students who never scored at 

or above the required ELP level on their state ELP test to be reclassified as fluent English 
proficient (RFEP) during the six years of the study (see, e.g., Olsen, 2010). RFEP students, 
in contrast to long-term ELL students, were defined as those ELL students who did score 

at or above the required ELP level on their state ELP test to achieve RFEP status. Struggling 
RFEP students were defined as those ELL students who met the ELP requirements for RFEP 
but did not pass the state ELA or reading content test at least once by the end of the sixth 
year of the study. We used the descriptor “Struggling” because this group of ELL students 
had moved from full success in their ELL support programs (i.e., passing their ELP test 

to achieve RFEP status) but had not yet achieved one of the minimum expected levels of 
success for all students (i.e., passing the state ELA or reading content test) during the six 
years of the study. This study also examines Transitioned RFEP students, who, in contrast to 
the Struggling RFEP students, were defined as RFEP students who passed the state ELA or 
reading content test at least once during the study period. 


The analytic sample for this study was purposefully limited to ELL students who had 
attended school for at least six years, in order to focus on the characteristics of ELL students 
who had participated in a state educational program for an extended period of time. 

The criteria for the analytic sample for each of the four ELL subgroups are described in 
figure 1. For each of the study’s four ELL student subgroups, we examined and compared 
the following characteristics: eligibility for free or reduced-price lunch (FRL), eligibil- 

ity for receipt of individualized education program (IEP) services, gender, and ELP level 
during the first year of the study. We analyzed and compared the patterns of characteris- 
tics of students within and across the three grade-level cohorts: the grade K cohort, who 
started kindergarten in 2006/07; the grade 3 cohort, who started grade 3 in 2006/07; and 
the grade 6 cohort, who started grade 6 in 2006/07. Specifically, we compared long-term 
ELL students versus all RFEP students, and we compared Struggling RFEP students versus 
Transitioned RFEP students. 
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Figure 1. Criteria for analytic sample, Utah, 2006/07-2011/12 


Was the student an ELL student 
in 2006/07? 


Did the student score proficient 
in the ELPA in 2006/07? 


Did the student exist in the data system 
from 2006/07 through 2011/12? 


Did the student have abnormal 
grade progress? 


Did the student have missing values 
in ELPA or ELA/reading test? 


Did the student score proficient in the 
ELPA from 2007/08 to 2011/12? 


YES 
Struaalin RFEP student-did the student pass 
RFEP erident No ELA/reading test at least once from 
2006/07 to 2011/12? 


ELPA is English language proficiency assessment. 


saan 


Source: Authors’ compilation. 


NO 
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Box 2. Utah’s testing process 


The Utah Academic Language Proficiency Assessment (UALPA) was developed to deter- 
mine English language learner (ELL) students’ progress in English language proficiency 
(ELP), in compliance with the NCLB Act of 2001. Prior to 2010/11, the five ELP pro- 
ficiency levels determined by the UALPA were Pre-Emergent, Emergent, Intermediate, Ad- 
vanced, and Fluent. During that time, the Utah State Department of Education (USOE) 
used both the UALPA and the English language arts criterion-referenced test (described 
below) to identify an ELL student’s level of English language fluency and to reclassify 
them as fluent English proficient (RFEP). To be reclassified, ELL students had to achieve 
an overall UALPA score of Advanced or above and an ELA content test score of Partial 
(level 2 of 4) or above, which is one level below passing (Sufficient, level 3 of 4). 


Starting in 2010/11, the UALPA began using new labels for the five ELP levels and also 
changing their reclassification assessment criteria. The new ELP level labels became 
Entering (formerly Pre-Emergent), Beginning (formerly Emergent), Developing (formerly 
Intermediate), Expanding (formerly Advanced), and Bridging (formerly Fluent). These 
two versions of Utah’s ELP levels are equivalent to each other. For clarity, this study uses 
the new ELP level labels. 


Beginning in 2010/11, the USOE has used only the UALPA to reclassify ELL students 
as fluent English proficient. If students achieve an overall score at the Bridging level, 
then they achieve RFEP and exit the ELL program. 


In Utah, the ELA content tests are grade-specific tests that assess the knowledge and 
skills of students in grades 2-115 in reading, writing, and listening—resulting in a single 
overall ELA score. The CRT has four proficiency levels: Minimal, Partial, Sufficient, and 
Substantial. To pass, students must score at the Sufficient level or above. 
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What we learned aE 


The study’s findings revealed key patterns across Utah’s three grade-level cohorts in relation 
to (1) the proportions of ELL students that met the study’s definitions of long-term ELL 
students, RFEP students, Transitioned RFEP students, and Struggling RFEP students; and 
(2) the characteristics of each of these subgroups of ELL students. 


Subgroup proportions 


The study examined the following questions in relation to ELL student subgroup 
proportions: 


» What proportion of ELL students fit the study’s definition of long-term ELL student 
by the end of the six years? 


» What proportion of ELL students fit the study’s definition of Struggling RFEP 
student by the end of the six years? 


Long-term ELL students and RFEP students 


By the end of the study period, at least two-thirds of the sample of ELL students scored at 
or above the required ELP level to meet RFEP criteria (table 1, gure 2). In other words, 
one-third or less of the ELL students were long-term ELL students at the end of the study 
period. The kindergarten cohort had the largest percentage of students who achieved RFEP 
(74 percent); 26 percent of the kindergarten cohort became long-term ELL students. 


Over 90 percent of the RFEP students in each grade-level cohort passed their ELA content 
test at least once (figure 2). Among long-term ELL students, at least 25 percent from each of 
the three grade-level cohorts passed the ELA content test at least once. Higher percentages 
of long-term ELL students passed the ELA content test in the grade 3 cohort (48 percent) 
and grade 6 cohort (42 percent) than in the kindergarten cohort (25 percent). Overall, a 
higher proportion of RFEP students passed their ELA content test compared to long-term 
ELL students. 
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Table 1. Less than one-third of ELL students became long-term 
ELL students, 2006/07-2011/12 


100 100 


Transitioned RFEP is reclassified as fluent English proficient and passed the English language arts content 
test at least once. Struggling RFEP is reclassified as fluent English proficient and never passed the English 
language arts content test during the study period. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah State Office of Education, for ELL students, 
2006/07 through 2011/12. 
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Figure 2. Overall, over 90 percent of RFEP students passed the 
ELA CRT, Utah, 2006/07-2011/12 


Grade 6: 965 


10°%o 


Grade 3: 1,975 


Grade K: 3,160 


MBLTELL MRFEP AIMS Passed AIMS Failed 


Note 1: RFEP is reclassification as fluent English proficient. K is kindergarten. CRT is English language arts 
criterion-referenced test. LTELL is long-term English language learner. 
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Note 2: The lower right-side circle in each grade cohort’s box represents the percentage of Struggling RFEP 
students (black wedge) and Transitioned RFEP students (light blue wedge). 


Note 3; Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education for ELL students, 2006/07 
through 2011/12. 


Struggling RFEP students and Transitioned RFEP students 


Across all three cohorts, over 90 percent of RFEP students became Transitioned RFEP 
students (see figure 2). In other words, less than 10 percent of the RFEP students remained 
as Struggling RFEP students at the end of the study. For example, out of the RFEP students 
in the kindergarten cohort, 7 percent were Struggling RFEP students and 93 percent were 
Transitioned RFEP students. 


Subgroup characteristics 


To understand the characteristics of each of the ELL student subgroups, the study examined 
the following questions: 


» What are the characteristics of long-term ELL students? How are these characteris- 
tics different from those of all RFEP students? 


» What are the characteristics of Struggling RFEP students? How are these charac- 
teristics different from those of the Transitioned RFEP students (that is, those who 
passed their ELA content test at least once by the end of the six years)? 


Long-term ELL students and RFEP students 


Overall and without exception, in each grade-level cohort a greater percentage of long-term 
ELL students compared to their RFEP peers were eligible for FRL (figure 3), were male 
(figure 4), began the study at one of the two lowest ELP levels (Entering or Beginning; 
figure 5), and were eligible for receipt of IEP services (figure 6). Further, the majority of 
long-term ELL students across all three grade-level cohorts were eligible for FRL (between 
64 and 82 percent; see figure 3) and were male (between 58 and 59 percent; see figure 4). 


Eligibility for free or reduced-price lunch 


In all three grade-level cohorts, the majority of long-term ELL and RFEP students (between 
57 to 82 percent) were eligible for FRL (see figure 3). A larger proportion of long-term ELL 
students were eligible for FRL compared to RFEP students. For example, in the kinder- 
garten cohort, 64 percent of long-term ELL students were eligible for FRL as compared to 
57 percent of RFEP students. 
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Figure 3. The majority of ELL students were eligible for free or 
reduced-price lunch, 2006/07 (percent) 
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RFEP is reclassification as fluent English proficient. K is kindergarten. FRL is free or reduced-price lunch. 
LTELL is long-term English language learner. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Gender 


Most of the long-term ELL students across all three grade-level cohorts were male (59 percent 
for the kindergarten cohort, 58 percent for the grade 3 cohort, and 59 percent for the grade 

6 cohort; see figure 4). Half or more of the RFEP students across all three grade-level cohorts 
also were male (50 percent for the kindergarten cohort, 53 percent for the grade 3 cohort, and 
56 percent for the grade 6 cohort). For each grade-level cohort, long-term ELL students had a 
greater percentage of male students than their RFEP student peers. 
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Figure 4. More than half of long-term ELL students were male, 
Utah, 2006/07 pea 


i Female Hi Male 


Grade K | Grade 3 | Grade 6 


RFEP is reclassification as fluent English proficient. K is kindergarten. LTELL is long-term English language 
learner. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Initial English language proficiency level 

For the grade 3 and grade 6 cohorts, the vast majority of ELL students—both those that 
became long-term ELL students and those that became RFEP students—had an initial 
ELP level of Developing (level 3 out of 5; see figure 5). The percentage of students in the 
grade 3 and grade 6 cohorts who started at the Expanding level (level 4) was much smaller 
(see figure 5) than in the kindergarten cohort. In the kindergarten cohort, the distribution 
of ELP levels was different than in the higher grade-level cohorts. The largest percentage 
(40 percent) of long-term ELL students were classified as Beginning (level 2), while the 
largest percentage (39 percent) of RFEP students were classified as Developing (level 3). 
Thirty-seven percent of the RFEP students were classified as Expanding (level 4). Across 
all three grade-level cohorts, the RFEP group had greater percentages of students initially 
at Developing (level 3) and Expanding (level 4) than did their long-term ELL peer group; 
correspondingly, the long-term ELL students in each grade-level cohort had greater 
percentages of students initially at the lowest two ELP levels (Entering and Beginning) 
compared to their RFEP peers. 


Regional Educational Laboratory West at WestEd 


MIL L/ 


MIL L/ 


Figure 5. The vast majority of ELL students in the grade 3 and 
grade 6 cohorts had an initial ELP level of Developing, 
Utah, 2006/07 (percent) 
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RFEP is reclassification as fluent English proficient. K is kindergarten. LTELL is long-term English language 
learner. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Eligibility to receive individualized education program services 


The percentage of long-term ELL students who were eligible to receive IEP services at the 
start of the study was greater in the higher grade-level cohorts that in the lower grade-level 
cohorts: 40 percent of long-term ELL students in the grade 6 cohort were eligible to receive 
IEP services, compared to 28 percent in the grade 3 cohort and 9 percent in the kinder- 
garten cohort (see figure 6). A higher percentage of long-term ELL students (between 9 
and 40 percent) than their RFEP student peers (between 4 and 14 percent) were eligible to 
receive IEP services. 
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Figure 6. Forty percent of long-term ELL students in grade 6 cohort 
were eligible to receive IEP services, Utah, 2006/07 (percent) 
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RFEP is reclassification as fluent English proficient. K is kindergarten. IEP is individualized education 
program. LTELL is long-term English language learner. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Struggling RFEP and Transitioned RFEP students 


With two exceptions, a greater percentage of Struggling RFEP students were eligible for 
FRL (figure 7), were male (figure 8), began the study at one of the two lowest ELP levels 
(Entering or Beginning) (figure 9), and were eligible to receive IEP services (figure 10), 
compared to their Transitioned RFEP student peers. Overall, the majority of the Struggling 
RFEP students across all three grade-level cohorts were eligible for FRL (see figure 7), were 
male (see figure 8), and started in one of the two highest ELP levels below Proficient (Basic 
or Intermediate) on their initial ELP assessment (see figure 9). Further, the vast majority 

of the Struggling RFEP students across all three grade-level cohorts were not eligible to 
receive IEP services (see figure 10). 


Eligibility for free or reduced-price lunch 


The vast majority of Struggling RFEP students were eligible for FRL (between 61 and 

86 percent; see igure 7). In the kindergarten and grade 3 cohorts, a higher percentage of 
Struggling RFEP students than Transitioned RFEP students were eligible for FRL (61 versus 
57 percent and 86 versus 77 percent, respectively). In contrast, in the grade 6 cohort, a 
slightly higher percentage of Transitioned RFEP students (77 percent) than Struggling 
RFEP students (75 percent) were eligible for FRL. 


Regional Educational Laboratory West at WestEd 16 


MIL L/ 


Figure 7. The majority of RFEP students were eligible for free or 
reduced-price lunch, Utah, 2006/07 (percent) 
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RFEP is reclassification as fuent English proficient. K is kindergarten. FRL is free or reduced-price lunch. 
Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Gender 


For the kindergarten cohort, 53 percent of Struggling RFEP students were male, while 

49 percent of Transitioned RFEP students were male (see figure 8). Similar proportions 
were also seen for grade 3 and grade 6 cohorts: a higher percentage of the Struggling RFEP 
group was male compared to the Transitioned RFEP group. Further, between 53 and 63 
percent of the Struggling RFEP students across all three grade-level cohorts were male. 
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Figure 8. More than half of Struggling RFEP students across all 
cohorts were male, Utah, 2006/07 (percent) 
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RFEP is reclassification as fluent English proficient. K is kindergarten. 
Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Initial English language proficiency level 

At the start of the study period, the highest percentage of students across all three cohorts 
in both the Struggling RFEP and Transitioned RFEP group were classified as Developing 
(level 3 of 5) on the ELP assessment (see figure 9). For example, in the grade 3 cohort, 

the vast majority of both Struggling RFEP students (82 percent) and Transitioned RFEP 
students (89 percent) started at the Developing level (see figure 9). At the same time, a 
greater percentage of Struggling RFEP students began the study at one of the two lowest 
ELP levels (Entering or Beginning) compared to their Transitioned RFEP peers. 
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Figure 9. The highest percentage of both Struggling RFEP and 
Transitioned RFEP students had an initial ELP level of 
Developing, Utah, 2006/07 (percent) 


i entering Wi Beginning WW Developing i Expanding 
© Trartoned CPJ 7 
fs) 
5  StruggingRFeP 27 
© Transitioned RFEP Aco 
7e) 
& Struggling RFEP |Z TENE 
“<< Transitioned RFEP TSS 
7e) 
je) 
a) Struggling RFEP iid 22 

0 10 20 30 40 50 60 #7 80 490 ~~ 100 


RFEP is reclassification as fluent English proficient. K is kindergarten. 
Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 


Eligibility to receive individualized education program services 


In the kindergarten cohort, both Struggling RFEP students and Transitioned RFEP students 
had the same percentage of students who were eligible to receive IEP services (4 percent; see 
figure 10). For the grade 3 and grade 6 cohorts, a higher percentage of Struggling RFEP 
students (25 and 19 percent, respectively) were eligible to receive IEP services compared to 
their Transitioned RFEP student peers (9 and 14 percent, respectively). 


Regional Educational Laboratory West at WestEd 


MIll/ 


Figure 10. A higher percentage of Struggling RFEP students received 
IEP services compared to Transitioned RFEP students in the 
grade 3 and grade 6 cohorts, Utah, 2006/07 (percent) 
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program. 


Note 1: Percentages might not add up to 100 due to rounding. 


Source: Authors’ analysis of student-level data from Utah Department of Education, for ELL students, 
2006/07 through 2011/12. 
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Implications and next steps aE 


This study describes the progress and characteristics of four groups of Utah’s ELL students— 
long-term ELL students, RFEP students, Struggling RFEP students, and Transitioned 
RFEP students—across three grade-level cohorts. The findings suggest three areas for 
further research. 


First, the percentages of ELL students who achieved the necessary ELP scores to be reclassi- 
fed as fluent English proficient (RFEP) during the study period remained close to constant 
across the three cohorts in Utah—from 74 percent in the kindergarten cohort to 67 percent 
in both the grade 3 and grade 6 cohorts. These results differ from the general findings 

of the research literature, which show that progressing in fluent English proficient status 

is generally more difficult for older ELL students than for younger ELL students (Cook, 
Wilmes, Boals, & Santos, 2008; Garcia, 2003). Conducting further research to examine 
possible reasons why this study’s ELP results differ from the general findings of the research 
literature could provide more accurate or targeted understandings of (1) why, in contrast to 
the general research findings, this study’s higher-grade-level ELL students were almost as 
successful as the lower-grade-level ELL students on the ELP test and (2) why between one 
quarter and one third of the ELL students still took longer than expected to reach English 
language proficiency. 


Second, the percentage of Struggling RFEP students in Utah remained closer to constant 
across the three grade-level cohorts than what might be expected based on the research liter- 
ature. The percentage of Struggling RFEP students in Utah was 7 percent in the kindergar- 
ten cohort, 6 percent in the grade 3 cohort, and then 10 percent in the grade 6 cohort. The 
results from Utah—a range from 6 to 10 percent across the three grade-level cohorts—appear 
to be at least somewhat inconsistent with the research literature that suggests that reclassified 
former ELL students generally have lower achievement in academic subjects in the higher 
grades, at least initially, than they do in the lower grades (Grissom, 2004; Kieffer, 2011, 
2010, 2008; Salazar, 2007). Further research designed to examine why Utah’s percentages 

of Struggling RFEP students in the grade 3 and grade 6 cohorts were lower than might be 
expected based on the research literature could provide insights into more effective practices 
for improving the achievement of RFEP students at the secondary level. 


Finally, across all cohorts, more long-term ELL students and Struggling RFEP students 
were eligible for FRL, eligible to receive IEP services, and male, compared to their more 
successful peers in the RFEP and Transitioned RFEP groups, respectively. These findings are 
consistent with the research literature on the negative relationship between poverty/eligi- 
bility for FRL and academic achievement (Berliner, 2006; Gandara, Rumberger, Maxwell- 
Jolly, & Callahan, 2003; Parrish et al., 2006; Rumberger & Gandara, 2004) and the nega- 
tive relationship between disabilities/eligibility for IEP services and academic achievement 
(Artiles, Rueda, Salazar, & Higareda, 2005; Samson & Lesaux, 2009; Sirin, 2005; Sullivan, 
2011; Vellutino et al., 1996). These findings are also consistent with studies on gender and 
its relationship to academic achievement, which have shown that boys are more likely than 
girls to have lower academic achievement in reading (Entwisle, Alexander, & Olson, 2007; 
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Perie et al., 2005; Robinson & Theule, 2011); however, no studies that specifically examined 
gender and academic achievement for ELL students have been located. Further research 
should continue exploring ways to assist these subgroups of ELL students (i.e., those who are 
eligible for FRL, those who are eligible to receive IEP services, and those who are male) to 
help them achieve greater academic success. 
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Limitations of the study Bil 


A major strength of this study is that Utah provided six years of data on all students who 
were in kindergarten, grade 3, and grade 6 in 2006/07 who had ELP test results and avail- 
able subject-matter content test results in ELA or reading. 


There are three limitations of this study. The first limitation relates to the scope of the 
sample. The study addresses the characteristics of three cohorts of ELL students who were 
in Utah’s data system for each school year over the six-school-year period of the study 
(2006/07-2011/12) and who advanced a grade level each year. Thus, the sample excludes 
mobile students who left and/or entered Utah during the study period. This stable ELL 
cohort was designed in response to the English Learner Alliance states’ interest in the 
performance of ELL students who had been in their school systems for an extended period 
of time, which, for this study, was defined as six years. As a result, the study sample is a more 
geographically stable group of ELL students than is present in most schools, so the profi- 
ciency rates and passing rates could be higher for these students than for the ELL popula- 
tion as a whole. The sample also excludes students who repeated or skipped a grade, due to 
difficulties tracking students who did not progress with the rest of their grade-level cohort. 
As a result, this sample does not include some ELL students who would be present in Utah’s 
schools on a given day. 


The percentages of students in the study samples, out of all of the ELL students in the 
cohort grade in the first year of each cohort, were 68 percent for the kindergarten cohort, 
65 percent for the grade 3 cohort, and 49 percent for the grade 6 cohort (table B1). 


The second limitation relates to comparisons among cohorts based on differences in sample 
characteristics and in content test-taking opportunities. For instance, there are likely to be 
differences in the characteristics of the students in the kindergarten, grade 3, and grade 6 
cohorts, especially related to initial ELP levels. For ELL students in the kindergarten cohort 
samples, kindergarten was their initial enrollment year in the state and, thus, their ELP level 
was their initial ELP level when they started school (which was when the study began for 
the kindergarten cohort). For ELL students in the grade 3 and grade 6 cohorts, grade 3 and 
grade 6 were their current enrollment grades, respectively, at the start of the study, and their 
ELP level was their current one at the start of the study; we do not know when these ELL 
students started in the state and we do not know their initial ELP level when they started 
school. Furthermore, the composition of the kindergarten, grade 3, and grade 6 cohorts 
could differ across cohorts due to differences in student mobility and grade repetition. 


Finally, the cohort samples could also be different from one another in that, compared to the 
kindergarten cohort, the grade 3 and grade 6 cohorts are likely composed of the students 
who have the most difficulty learning English, since faster learners will have already 
achieved RFEP; in kindergarten, these faster English-language learners will have not yet 
been given a chance to achieve RFEP. Differences in ELL student achievement percentages 
across the three cohorts are also likely influenced by differences in the content of the tests 
because Utah, like the other states in the English Learner Alliance, increases the difficulty 
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of its ELP and ELA content tests as the grade level increases. Further, students in the 
kindergarten cohort have fewer opportunities to take, and therefore have fewer opportuni- 
ties to pass, Utah’s ELA content test, as it is first administered in grade 3. However, while 
these factors are all limitations of the study, they also reflect the actual experience of ELL 
students, over time, in these state systems. 
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Appendix A. Methodology aE 


This study is a descriptive analysis of the characteristics of four categories of English 
language learner (ELL) students in Utah—long-term English language learner students, reclas- 
sified fluent English proficient (RFEP) students, Struggling RFEP students, and Transitioned 
RFEP students. In this study (and in the two companion reports on ELL students in Arizona 
and Nevada), long-term ELL students were defined as students who never scored at or above 
the required English language proficiency (ELP) level on their state ELP test to be reclassi- 
fed as fluent English proficient (RFEP) during the six years of the study. In contrast to the 
long-term ELL students, RFEP students were defined as those ELL students who did score 
at or above the required ELP level on their state ELP test to achieve RFEP status. Struggling 
RFEP students (part of federal Title II accountability in annual measurable achievement 
objective 3) were defined as those ELL students who met the ELP requirements for reclas- 
sification as fluent English proficient but did not pass the state ELA or reading content 

test by the end of the six-year study period. In contrast to the Struggling RFEP students, 
Transitioned RFEP students were defined as RFEP students who passed the state ELA or 
reading content test at least once during the study period. 


Data sources 


The data source for this study was Utah’s statewide data system. All of the data files used in 
the study contained student-level information from school year 2006/07 through school year 
2011/12. Each file had anonymous, unique student identification numbers, which allowed 
for matching of students across different files. The data were analyzed in three parallel 
grade-span cohorts: kindergarten through grade 5, grade 3 through grade 8, and grade 6 
through grade 11. 


Methods 


The study used descriptive statistics to describe patterns of characteristics of long-term ELL 
students and RFEP students, with particular attention to those RFEP students who did not 
pass their ELA or reading content tests during the study period (referred to in this study as 
Struggling RFEP students). 


The analysis resulted in tables and figures that compare the percentages of certain charac- 
teristics of long-term ELL students to those of RFEP students and compare the percent- 
ages of certain characteristics of Struggling RFEP students to those of Transitioned RFEP 
students. The study focused on the following ELL student characteristics: eligibility for free 
or reduced-price lunch (FRL), eligibility for receipt of individualized education program 
(IEP) services, gender, and initial ELP level. These characteristics were characteristics 
that students possessed during the first year of the study, 2006/07. The study focused on 
these particular characteristics because they were identified by research literature as being 
linked with long-term ELL status and low academic achievement in general, and because 
data on these characteristics were available in the state data system. Differing patterns of 
student achievement based on gender have also been described for ELL students, as well 
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as for students in general (Dunn, Griggs, & Price, 1993; Entwisle, Alexander, & Olson, 
2007; Restak, 1979). Overall, eligibility for FRL and learning disabilities (such as a reading 
disability) that result in eligibility for IEP services have consistently been found to be highly 
associative with student achievement (Sirin, 2005; Vellutino et al., 1996). 


Criteria for analytic sample 


Students were included in the analytic sample based on meeting all of the following criteria 
(the same criteria were used across all three companion reports): 


» Had an initial ELP level lower than Proficient in school year 2006/07. 


» Were in the agency’s data system for all six years of the study (i.e., in the data files 
of each school year, from 2006/07 to 2011/12). 


» Began in the cohort grade (K, 3, or 6) in school year 2006/07, with normal grade 
progress through school year 2011/12.* 


» Achieved the ELP level required for RFEP status during the study period or took the 
ELP test in the last school year, 2011/12. 


» Passed the state ELA or reading content test at least once, or had a state ELA or 
reading content test result in each year, when available, during the study period.° 


Please note that the analytic samples in each grade-level cohort included all ELL students 
who met all of the above criteria, not just ELL students who were new to the state in 
2006/07. 
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Appendix B. Details on 
analytic samples 


Table B1. Steps to get analytic samples in Utah 


Start point English language learner stu- 
dents in 2007 (initial ELP level 4,678 3,048 1,969 
lower than Bridging) 

Step 1 Students excluded because not 1,015 
present in all 6 years 


Students excluded because of 
abnormal grade progress 


Students excluded because of 
missing values 


Analytic sample 


ELL is English language learner. ELP is English language proficiency. K is kindergarten. 


Note 1: Percentages may not add up to 100 percent due to rounding. 


Source: Authors’ analysis of student-level data from Utah State Office of Education, 2006/07 through 2011/12. 
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Table C1. Characteristics of long-term ELL students and RFEP students for 
kindergarten, grade 3, and grade 6 cohorts, Utah 


Grade K cohort Grade 3 cohort | Grade 6 cohort 


Long-term Long-term Long-term 
FEILIL RFEP Overall ELL RFEP Overall ELL RFEP Overall 
Characrrsics 


Gender 


TEP in 2007 


[Unknown [OY oo] of oo] of oof of eof of eof of of of oo] of oo] of an] 
Pie ef 92] [a2] ep ssf sf ma) ef oa] sofa] ofa] ox es] 2] 2] 


FRL in 2007 


FRL 


Initial ELP 
level in 2007 


100.0 3,160 100.0 ! F ! 644 100.0 965 100.0 


ELL is English language learner. ELP is English language proficiency. FRL is free or reduced-price lunch. IEP is individualized education 
program. K is kindergarten. RFEP is reclassified fluent English proficient. 


Note 1; Percentages may not add up to 100 percent due to rounding. 


Source: Authors’ analysis of student-level data from Utah State Office of Education, 2006/07 through 2011/12. 
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Table C2. Characteristics of Struggling RFEP students and Transitioned RFEP 
students who passed their ELA content test for kindergarten, grade 3, and 
grade 6 cohorts, Utah 


RFEP in grade K cohor 
Characertes | nonter | tocar [number | recon | winter | recor | nimte | recor [rim | tocar | wanter | recor _| 
Gender 
IEP in 2007 
ee ST 
FRL in 2007 
| unknown [of eo] of oo] of oo] of oo ooo ot 
Cee ||| ose Go| ar] | | |e | 


Initial ELP level 
in 2007 


Pasgnning | f3[ esr] | ef of asf sf nf | 
[pecking | 6] 9 ws 9] a] wa] ura] waf af 9 a] va 
P espending | 36 of val a] 2] asf] i] 


54 
Total 167 100.0 Dell 5) 100.0 iS 100.0 1,246 100.0 67 100.0 a7 100.0 


ELP is English language proficiency. FRL is free or reduced-price lunch. IEP is individualized education program. K is kindergarten. 
RFEP is reclassified fluent English proficient. Struggling RFEP students are those RFEP students who did not pass the ELA content 
test during the course of the study. Transitioned RFEP students are those RFEP students who passed the ELA content test at least once 
during the course of the study. 


Note 1: Percentages may not add up to 100 percent due to rounding. 


Source: Authors’ analysis of student-level data from Utah State Office of Education, 2006/07 through 2011/12. 
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Notes 
————————— ee 


1 The ELL student characteristics reported in this study—eligibility for free or reduced- 
price lunch, eligibility to receive individualized education program services, and English 
language proficiency level—are characteristics that students possessed during the first year 
of the study, 2006/07. 


2 For Nevada, the analysis was done using data on ELL students from the state’s two larg- 
est school districts: Washoe County and Clark County. 


3 Prior to 2008, Utah administered its CRT in grades 1 through 11. In spring 2008, Utah 
began administering its CRT in grades 2 through 11. And, in spring 2011, Utah began 
administering its CRT in grades 3 through 11. For the study period, which began in 
2006/07, only the kindergarten cohort was affected. Student progress on the CRT in the 
kindergarten cohort will be described beginning in grade 2 in spring 2009. 


4 Students who repeated or skipped a grade, and thus no longer progressed with the vast 
majority of peers in their grade-level cohort, were excluded due to difficulties in tracking 
these students once they were out of step with their cohort. 


5 All ELL students in these states were required to take their ELA/reading content tests 
each year the tests were administered. ELL students were not exempted from this testing 
requirement. 
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